SECTION 2: MEASUREMENT Risk Adjustment for Quality Improvement

We can learn what is achievable with current technologies by comparing our neonatal intensive care unit outcomes with others. Because neonatal intensive care units may vary with respect to their case-mix, risk adjustment is essential to making fair comparisons in any research that does not equalize risks through randomization. Risk adjustment first requires strict definition of each specific outcome. Then each risk factor is measured and weighted accordingly. Severity of illness scores are a special form of risk adjustment. The leading newborn illness severity scores rely on physiology-based items from bedside vital signs and laboratory tests. The mechanics of score development are discussed including item selection, definition, collection, and potential biases. The process of weighting risk factors usually involves building multivariate models. Issues of derivation, validation, discrimination, calibration, and reliability affect the utility of all scores. Once a comparison is appropriately risk-adjusted, there are important cautions about interpretation, including the source of the reference (benchmark) population, sample size, and biases from incomplete risk adjustment. Nonetheless, these findings can spur quality improvement efforts that can lead to dramatic, system-wide improvements in outcomes. Pediatrics 1999;103:255–265; severity of illness, pediatrics, neonatology, intensive care units, neonatal intensive care, research ethics, health services research, outcomes research, benchmarking. ABBREVIATIONS. SNAP, Score for Neonatal Acute Physiology; CRIB, Clinical Risk Index for Babies; NICU, neonatal intensive care unit; RDS, respiratory distress syndrome; ECMO, extracorporeal membrane oxygenation; APACHE, Acute Physiology and Chronic Health Evaluation; ICU, intensive care unit; PRISM, Pediatric Risk of Mortality. The purpose of this article is to provide a general overview on the measurement of illness severity and methods of risk adjustment used for neonatal intensive care. This is not intended as a primer for developing an illness severity score. Definitive treatments of risk adjustment methodologies are available,1,2 to which we acknowledge our intellectual debt. Other very good discussions of methodology are included with the descriptions of adult and pediatric intensive care scoring systems3–12 and a review of adult scores13 as well as the original descriptions of the Score for Neonatal Acute Physiology (SNAP)14,15 and the Clinical Risk Index for Babies (CRIB).16 Our two previous reviews of neonatal illness severity focused first on the variety of neonatal outcomes scores17 and then on the range of applications of SNAP and CRIB in their first 5 years of use.18 Our focus here is on the mechanics of risk adjustment, the shortcomings inherent in the design of any risk adjustment strategy, and the application of these to inter-neonatal intensive care unit (NICU) comparisons for benchmarking, quality improvement, and health services research. PURPOSES OF RISK ADJUSTMENT